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ABSTRACT 

This guide presents ways for practitioners to use 
Chapter 1 achievement data tc better understand and evaluate elements 
of a Chapter 1 program. A variety of types and levels of questions 
can be addressed with achievement data. Combining standardized 
achievement test scores with data from the program and from the 
regular classroom greatly increases the range of questions that can 
be answered. Full group data alxow evaluation of: (1) how program 
results compare to district, state, or national results in each 
subject area; (2) what the long-term effects of the Chapter 1 program 
are; (3) how the standardized test results compare to performance on 
program criterion measures; and (4) what the trends in achievement 
over time for each cohort are. Subgroup data allow the determination 
of the relative gains for students at different levels, and 
assessment of how students who exit the program compare with those 
who remain. Individual data can provide insights into how to deal 
with students who do not seem to progress. Case study analysis can 
reveal much about the achievement of individuals. Data are generally 
collected with a specific purpose, but this guide shows ways in which 
additional questions and hypotheses can be examined. Five graphs and 
three tables present the study results. (SLD) 
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PROFILING CHAPTER 1 
ACHIEVEMENT DATA 



In this short guide you will find examples of ways you can use your 
Chapter 1 achievement data to help you better understand and evaluate some 
elements of your program. Data are generally collected with a specific purpose 
in mind, stated as a research question or statement of hypothesis. However, once 
the data are collected and examined, additional questions or hypotheses may 
arise. These examples represent only some of the possible ways you could look at 
your data and are meant to help stimulate ideas of your own. 

This guide does not include information about how to select your 
instruments, or display or interpret data. For help with those tasks, see the "Guide 
for Selecting Instruments for Profiling" the "Guide for Reporting and/or Presenting 
Profile Data? and the "Guide for Interpreting Profile Data:' In addition, you will 
find helpful information on how to formulate questions and design your profile 
in the "Guide for Determining the Profile Purpose." 



Sources of Data 



A variety of types and levels of questions can be addressed with 
achievement data. Some questions will require analysis of averages for the full 
group or a subgroup, and some Questions will best be answered by a case study 
approach. Full group averages focus attention on trends in achievement for the 
entire program. Subgroup averages show patterns for groups of students with 
common characteristics (such as level of pretest score) that may be hidden by 
whole group averages. Case studies can provide insights into program impact by 
taking an indepth look at a few individuals (i.e. students progressing at a slower 
rate than most of the others, or students who progress faster with different 
instructional methods). 

The range of questions you can answer wili depend on how many sources 
of data you have. By combining standardized achievement test scores with data 
from the program and from the regular classroom such as class grades, number 
of objectives mastered, GPA, and scores on program-based criterion measures, 
you can learn much more. Standardized test scores by themselves are not a real 
reliable source of information, especially if your group is small. However, when 
combined with other sources of information, they are an adequate source of 
information. As a genecal rule, the more sources of information you can get to 
answer a particular question, the more confidence you can have in your 
conclusions. 
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Examples of Profiting Achievement Data 



Full Group Data 



Using group data as it is from the annual Chapter 1 report, you could 
answer questions like the following: 

• How do the program results compare overall to district, state, or national 
results in each subject area (see Hgs. 1 and 2)? 

• What are the long term effects of the Chapter 1 program (see Fig. 3)? 

The line graph in Figure 1 shows that in reading, 'he district exceeds both 
the state and the nation in almost all grades. Figure 2 shows that the district is 
performing slightly below the state in math, but still above the nation at most 
grade levels. 



Chapter 1 Achievement 

Reading, Fall-Spring Testing 




District 

State 

Notion 



Grade 



Figure 1. Trends in reading achievement for the district, state, and the nation 
on a fall to spring testing cycle. 
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Figure Z Trends in math achievement for the district, state, and the nation 
on a fall to spring testing cycle. 

The bar graph in Figure 3 shows grades 2-10 gains in Chapter 1 reading for 
school years 1983/84 through 1985/86. Note that the fluctuation in the size of the 
gains within grades is relatively small, and that gains are much larger in the 
early grades than in the later grades. 

Chapter 1 Reading Program 

Gains for 1983/84 to 1985/86 
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Figure 3. Comparison of NCE gains over three years for grades 2 through 10. 



ERIC 



With data from your Chapter 1 annual report combined with some 
additional data from the district or from classroom teachers, you could address 
questions such as: 

• How do the standardized test results compare to student performance in 
the program-i.e. performance on program criterion measures (see Table 1)? 

• What are the trends in achievement over time for each cohort (graduating 
class) (see Table 2)? 

Table 1 shows essentially two pretest-posttest comparisons of Chapter 1 
students' performance. In the first three columns are the pretest and posttest 
scores and NCE gains for each grade, and in the last two columns you 1l find the 
average grade (from a 61 which represents failing grade to a 78 which represents 
a C) from the first grade reporting period in the fall and the last grade reporting 
period in the spring. The extent to which the gains are parallel suggests that the 
Chapter 1 program is affecting both the general achievement level of students 
and performance in the regular school program. The small gain or loss in grades 
compared to a relatively large gain i.i achievement in grades 3 and 4 is a finding 
that needs further investigation. 



NCE Gains and Gains in Class Grades 
for Grades 2 Through 8 

NCE score Grade 
Grade Pre Post Gain Pre Post 



2 


28 


35 


7 


62 


78 


3 


31 


46 


15 


61 


65 


4 


36 


45 


9 


65 


63 


5 


28 


34 


6 


71 


76 


6 


29 


33 


4 


68 


75 


7 


30 


35 


5 


70 


73 


8 


26 


30 


4 


63 


70 



Tabic 1. Comparison of achievement test gains with gains in class grades 
throughout the year. 



Table 2 shows the fall and spring achievement test scores (from the district 
wide achievement test) of students who were in Chapter 1 in grades 1-3 in 1982. 
By 1986, of the 40 students who were originally in the program, 8 are still in 
Chapter 1. Once a child is exited from Chapter 1 s/he rarely re-enters the 
program. 
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Pretest and Posttest NCE Scores 
For Students Beginning Grades 1-3 in 1982 



Grades 1-3 
Student 

George 

Sean 

Alice 

Theresa 

John C. 

Travis 

Andrew 

Felicia 

Carole 

Stephanie 

Maria G. 

Joel* 

Amanda 

Trevor 

Annie 

Rhoda 

Steven 

Michael 

Randy 

Jamie 

Maria L. 

Beth 

Robbie 

Kelly 

Christie Ann 
James N. 
Gordon 
John P. 
Juanita 
Su Lin 
Hon Lee 
Maria P. 
Tiffany Ann 
Peter* 
Jerrod 
Kimberly 
Cathy 
Angie 
Shannon* 
Scott 



1982/83 
Pre Post 



1983/84 
Pre Post 



1984/85 
Pre Post 



1985/86 
Pre Post 



13 

42 

43 

35 

20 

35 

38 

21 

22 

26 

19 

29 

40 

25 

23 

18 

22 

34 

23 

39 

41 

43 

35 

28 

15 

17 

20 

38 

25 

29 

22 

40 

34 

21 

18 

39 

25 

38 

18 

40 



ex- 



20 
41 
49 
38 
27 
42 
36 
28 
2G 
31 
30 
40 

moved 
24 
30 
26 
25 
41 
27 
38 
43 

48 ex- 
43 
35 
28 
23 
27 
42 
33 
26 
29 
43 
33 
36 

moved 
44 



17 28 
43 46 ex- 

46 47 

34 39 
22 28 
41 47 ex- 
43 44 ex- 
moved 

24 30 
30 34 
26 35 
41 48 ex 



25 36 moved — 

■47 — 48 46 47 

-45 46 47 44 

35 47 ex-42 40 

27 33 32 38 

■45 47 43 41 

-42 moved-— 



32 



37 



31 34 

moved 

30 41 ex-40 41 

-50 53 51 55 



27 36 

28 33 
23 39 
22 25 

40 46 

moved 

36 44 ex 

41 45 ex 
46 47 

40 47 ex 

32 39 
2 5 moved- 
20 31 
moved 

41 47 ex 

moved 

28 37 
26 34 
41 moved- 

33 39 
28 33 



34 


40 


38 


43 


31 


35 


30 


38 


40 


49 


ex-4 5- 


— 48 


25 


28 


21 


23 









■41- 
■41- 
-43- 
■50- 
38 



-40- 
-44- 
—45- 
—51- 
39 



•38 41 

■39 42 

■40 41 

■48 47 

4 2 ex41 



28 

•46- 



34 
-45 



moved 



42 44 



38 
30 



41 

39 



36 
43 



39 
50 



37 
30 



34 
31 



moved — 
27 26 



39 
46 
40 
43 



ex 



41 moved- - 

moved • 

-43 49 

37 43 ex- 
42 moved— 



•47 48 44 49 

•39 36re en34 40 



Table 1. NCE scores across four years for students who 
began grades 1-3 in 1982. Scores are from the 
district-wide achievement testing so that there 
would be a record for students exited from the 
chapter 1 program. 
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Subgroup Data 



Further processing of the data-for the most part very simple procedures- 
will allow you to examine your program more deeply. By so doing, you could 
answer questions like: 

• Are gains the same for those far below grade level as for those moderately 
below? 

Figure 4 shows that gains are greater for those below the 30th percentile 
on the pretest, while those at the 30tn percentile or above show smaller gains. 
Note, however, that in most cases it is those that are in the higher group that 
reach grade level and can be exited from the program. Those who started out 
below the 30th percentile are not likely to reach grade level in one year. 

Chapter 1 Reading Program 



Gains fcr Students with NCE Scores Below and Above 30 




Pretest 
Posttest 



Beto* 30 30 end Above 

Pretest Score 



Figure 4. Comparison of NCE gains for students scoring below 30 on the 

pretest and for students scoring 30 and above in a Chapter 1 reading 
program. 

• How do students who exit the program after one year compare with 
students who remain in the program for two, three, or four years? 

As shown in Figure 5, students who reached the exit criterion for the 
program after one year had high pretest scores relative to those who remained in 
the program three or four years. Again it is apparent that those who are closer 
to grade level when they enter the program have a much better likelihood of 
reaching the exit criterion. Often, those who remain in the program three years 
or longer will not show gains as great even as those in Figure 5. 



Chapter 1 Reading Program 

Comparison of Gains for Students In the Program 1, 2, or 3 Years 
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Figure 5. Comparison of fall to spring gains for students who were in a 

Chapter 1 Reading program for 1 year, 2 years, 3 years, or 4 years. 

Individual Data 

Analyses for individual students could provide insights into how to deal 
with students who don't seem to progress for reasons such as differences in 
learning style. For example, students who show unusual gains or losses, students 
who show drastic changes in gains or losses over time, or students who have been 
in the program for three years or more would be good choices for case study 
analysis. 

Sample questions might be: 

• Is the student's classroom performance similar to performance on the 
standardized test? 

• Is the student's pattern of progress stable? 

• Does the student perform similarly across subject matter areas? 

Referring back to Table 2, you'll notice that Peter was one of only 8 
students who remained in the program all four years. He started out well below 
grade level, had good gains the first two years, and then began to decline. An 
examination of some other indicators of Peter's achievement may give some 
insight into why he has such a pattern. In Table 3 you will find a composite of 
the following achievement measures for Peter. 1) NCE scores, 2) expanded 
standard scores, 3) Peter's first and last six-week grades in his regular class, and 4) 
scores on criterion tests for the Chapter 1 program. 
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Case Study Profile for Peter 
School Years 1982/83 to 1985/86 





1982/83 


1983/84 
F S 


1984/85 


1985/86 
r 9 


NCE Scores 


22 36 


28 33 


30 31 


27 26 


Expanded 


387 523 


512 563 






Class 
Grade 


53 65 
(F) (D) 


66 75 
(D) (C-) 


78 68 
(C) (D+) 


60 61 
(D-) (D-) 


Criterion 
Tests (% 
Passed) 


30 60 


70 75 


75 78 


79 80 


*F=Fall **S=Spring 









Tabic 3. Comparison of different measures of achievement for a student in 
Chapter 1 reading. 



Peter was in second grade in 1982-83. Though the NCE scores show an 
overall drop in achievement from grade 2 to grade 5, the expanded standard 
scores-scores that indicate Peter's growth over time rather than how he 
compares with his classmates-show that his growth leveled off somewhat when 
he reached the end of third grade. From that point his growth continues to be 
very gradual. His grades in the regular school program follow the same pattern 
as the NCE scores, but according to the criterion tests for Chapter 1, he is making 
good progress. What could be causing this pattern? In the reading curriculum, it 
is in the third grade that the emphasis shifts from the very mechanical reading 
skills of decoding, phonics, and learning sight words to an emphasis on 
comprehension. Peter was making progress with the mechanics of reading, but 
he apparently h-s not been able to generalize from the mechanics of reading to 
reading for comprehension. In the Chapter 1 program, he continues to receive 
remedial help with mechanics, but little time is spent on helping Peter learn to 
use those skills to help him function at the same level as his classmates in the 
regular classroom setting. 

Other students who would be good candidates for case study analysis 
(referring again to Table 2) are Joel and Shannon. Joel shows phenomenal 
growth in this four year period. Even when he is exited from the Chapter 1 
program, he continues to show higher achievement scores. Perhaps we could 
learn something by studying Joel that would help to duplicate his pattern in 
other students. Shannon, on the other hand, shows good growth as long as she 
receives the extra help from Chapter 1, but when she is exited from the program 
her achievement scores decline to the point that she must re-enter the program. 
Studying Shannon may provide help in avoiding a similar pattern in other 
students. 
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Summary 

As was mentioned earlier, the examples in this guide are meant only to 
stimulate you to think of ways of using your data to answer your tjuestions. They 
are not meant to be copied exactly. It takes some creativity and insight to 
determine what questions you should address and what data you need to answer 
them. Make it a matter of thought and group discussion. Once you have 
completed those steps and have assembled your data, you are ready to analyze it, 
interpret your findings, and report them. This information can then be used in 
setting goals and determining what improvements need to be made in your 
program. 
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